Covariate shift estimation based adaptive ensemble learning for handling non-stationarity in motor imagery related EEG-based brain-computer interface.

The non-stationary nature of electroencephalography (EEG) signals makes an EEG-based brain-computer interface (BCI) a dynamic system, thus improving its performance is a challenging task. In addition, it is well-known that due to non-stationarity based covariate shifts, the input data distributions of EEG-based BCI systems change during inter- and intra-session transitions, which poses great difficulty for developments of online adaptive data-driven systems. Ensemble learning approaches have been used previously to tackle this challenge. However, passive scheme based implementation leads to poor efficiency while increasing high computational cost. This paper presents a novel integration of covariate shift estimation and unsupervised adaptive ensemble learning (CSE-UAEL) to tackle non-stationarity in motor-imagery (MI) related EEG classification. The proposed method first employs an exponentially weighted moving average model to detect the covariate shifts in the common spatial pattern features extracted from MI related brain responses. Then, a classifier ensemble was created and updated over time to account for changes in streaming input data distribution wherein new classifiers are added to the ensemble in accordance with estimated shifts. Furthermore, using two publicly available BCI-related EEG datasets, the proposed method was extensively compared with the state-of-the-art single-classifier based passive scheme, single-classifier based active scheme and ensemble based passive schemes. The experimental results show that the proposed active scheme based ensemble learning algorithm significantly enhances the BCI performance in MI classifications.

Classification of propofol-induced sedation states using brain connectivity analysis.

connectivity measurements can provide key information about ongoing brain processes. In this paper, we propose to investigate the performance of the binary classification of Propofol-induced sedation states using partial granger causality analysis. Based on the brain connectivity measurements obtained from EEG signals in a database that contains four sedation states: baseline, mild, moderate, and recovery, we consider eight sensors and evaluate the area under the ROC curve with five classifiers: the k-nearest neighbor (density method), support vector machine, linear discriminant analysis, Bayesian discriminant analysis, and a model based on extreme learning machine. The results support the conclusion that the different Propofol-induced sedation states can be identified with an AUC of around 0.75, by considering signal segments of only 4 second. These results highlight the discriminant power that can be obtained from scalp level connectivity measures for online brain monitoring.

Current source density estimates improve the discriminability of scalp-level brain connectivity features related to motor-imagery tasks.

Recent progress in the number of studies involving brain connectivity analysis of motor imagery (MI) tasks for brain-computer interface (BCI) systems has warranted the need for pre-processing methods. The objective of this study is to evaluate the impact of current source density (CSD) estimation from raw electroencephalogram (EEG) signals on the classification performance of scalp level brain connectivity feature based MI-BCI. In particular, time-domain partial Granger causality (PGC) method was implemented on the raw EEG signals and CSD signals of a publicly available dataset for the estimation of brain connectivity features. Moreover, pairwise binary classifications of four different MI tasks were performed in inter-session and intra-session conditions using a support vector machine classifier. The results showed that CSD provided a statistically significant increase of the AUC: 20.28% in the inter-session condition; 12.54% and 13.92% with session 01 and session 02, respectively, in the intra-session condition. These results show that pre-processing of EEG signals is crucial for single-trial connectivity features based MI-BCI systems and CSD can enhance their overall performance.

Towards Optimization of Gaze-Controlled Human-Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients.

Virtual keyboard applications and alternative communication devices provide new means of communication to assist disabled people. To date, virtual keyboard optimization schemes based on script-specific information along with multimodal input access facility are limited. In this work, we propose a novel method for optimizing the position of the displayed items for gaze-controlled tree-based menu selection systems by considering a combination of letter frequency and command selection time. The optimized graphical user interface (GUI) layout has been designed for a Hindi language virtual keyboard based on a menu wherein 10 commands provide access to type 88 different characters along with additional text editing commands. The system can be controlled in two different modes: eye-tracking alone and eye-tracking with an access soft-switch. Five different keyboard layouts have been presented and evaluated with ten healthy participants. Further, the two best performing keyboard layouts have been evaluated with eye-tracking alone on ten stroke patients. The overall performance analysis demonstrated significantly superior typing performance, high usability (87% SUS score), and low workload (NASA TLX with 17 scores) for the letter frequency and time-based organization with script specific arrangement design. This work represents the first optimized gaze-controlled Hindi virtual keyboard, which can be extended to other languages.

Toward Optimization of Gaze-Controlled Human-Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients.

Virtual keyboard applications and alternative communication devices provide new means of communication to assist disabled people. To date, virtual keyboard optimization schemes based on script-specific information, along with multimodal input access facility, are limited. In this paper, we propose a novel method for optimizing the position of the displayed items for gaze-controlled tree-based menu selection systems by considering a combination of letter frequency and command selection time. The optimized graphical user interface layout has been designed for a Hindi language virtual keyboard based on a menu wherein 10 commands provide access to type 88 different characters, along with additional text editing commands. The system can be controlled in two different modes: eye-tracking alone and eye-tracking with an access soft-switch. Five different keyboard layouts have been presented and evaluated with ten healthy participants. Furthermore, the two best performing keyboard layouts have been evaluated with eye-tracking alone on ten stroke patients. The overall performance analysis demonstrated significantly superior typing performance, high usability (87% SUS score), and low workload (NASA TLX with 17 scores) for the letter frequency and time-based organization with script specific arrangement design. This paper represents the first optimized gaze-controlled Hindi virtual keyboard, which can be extended to other languages.

A review of rapid serial visual presentation-based brain-computer interfaces.

Rapid serial visual presentation (RSVP) combined with the detection of event-related brain responses facilitates the selection of relevant information contained in a stream of images presented rapidly to a human. Event related potentials (ERPs) measured non-invasively with electroencephalography (EEG) can be associated with infrequent targets amongst a stream of images. Human-machine symbiosis may be augmented by enabling human interaction with a computer, without overt movement, and/or enable optimization of image/information sorting processes involving humans. Features of the human visual system impact on the success of the RSVP paradigm, but pre-attentive processing supports the identification of target information post presentation of the information by assessing the co-occurrence or time-locked EEG potentials. This paper presents a comprehensive review and evaluation of the limited, but significant, literature on research in RSVP-based brain-computer interfaces (BCIs). Applications that use RSVP-based BCIs are categorized based on display mode and protocol design, whilst a range of factors influencing ERP evocation and detection are analyzed. Guidelines for using the RSVP-based BCI paradigms are recommended, with a view to further standardizing methods and enhancing the inter-relatability of experimental design to support future research and the use of RSVP-based BCIs in practice.

A multimodal interface to resolve the Midas-Touch problem in gaze controlled wheelchair.

Human-computer interaction (HCI) research has been playing an essential role in the field of rehabilitation. The usability of the gaze controlled powered wheelchair is limited due to Midas-Touch problem. In this work, we propose a multimodal graphical user interface (GUI) to control a powered wheelchair that aims to help upper-limb mobility impaired people in daily living activities. The GUI was designed to include a portable and low-cost eye-tracker and a soft-switch wherein the wheelchair can be controlled in three different ways: 1) with a touchpad 2) with an eye-tracker only, and 3) eye-tracker with soft-switch. The interface includes nine different commands (eight directions and stop) and integrated within a powered wheelchair system. We evaluated the performance of the multimodal interface in terms of lap-completion time, the number of commands, and the information transfer rate (ITR) with eight healthy participants. The analysis of the results showed that the eye-tracker with soft-switch provides superior performance with an ITR of 37.77 bits/min among the three different conditions (p<;0.05). Thus, the proposed system provides an effective and economical solution to the Midas-Touch problem and extended usability for the large population of disabled users.

Propofol-induced sedation diminishes the strength of frontal-parietal-occipital EEG network.

The level of conscious experience can be effectively and reversibly altered by the administration of sedative agents. Several studies attempted to explore the variations in frontal-parietal network during propofol-induced sedation. However, contradictory outcomes warrant further investigations. In this study, we implemented the Neural Gas algorithm-based delay symbolic transfer entropy (NG-dSTE) for investigation of frontal-parietal-occipital (F-P-O) network using scalp EEG signals recorded during altered levels of consciousness. Our results show significant disruption of the F-P-O network during mild and moderate levels of propofol sedation. In particular, the interaction between frontal and parietal-occipital region is highly disturbed. Moreover, we found measurable effect of sedation on local interactions in the frontal network whereas parietal-occipital network experienced least variations. The results support the conclusion that the connectivity based features can be utilized as reliable biomarker for assessment of sedation levels effectively.

Current Source Density Estimation Enhances the Performance of Motor-Imagery-Related Brain-Computer Interface.

The objective is to evaluate the impact of EEG referencing schemes and spherical surface Laplacian (SSL) methods on the classification performance of motor-imagery (MI)-related brain-computer interface systems. Two EEG referencing schemes: common referencing and common average referencing and three surface Laplacian methods: current source density (CSD), finite difference method, and SSL using realistic head model were implemented separately for pre-processing of the EEG signals recorded at the scalp. A combination of filter bank common spatial filter for features extraction and support vector machine for classification was used for both pairwise binary classifications and four-class classification of MI tasks. The study provides three major outcomes: 1) the CSD method performs better than CR, providing a significant improvement of 3.02% and 5.59% across six binary classification tasks and four-class classification task, respectively; 2) the combination of a greater number of channels at the pre-processing stage as compared with the feature extraction stage yields better classification accuracies for all the Laplacian methods; and 3) the efficiency of all the surface Laplacian methods reduced significantly in the case of a fewer number of channels considered during the pre-processing.

Single-trial effective brain connectivity patterns enhance discriminability of mental imagery tasks.

OBJECTIVE: The majority of the current approaches of connectivity based brain-computer interface (BCI) systems focus on distinguishing between different motor imagery (MI) tasks. Brain regions associated with MI are anatomically close to each other, hence these BCI systems suffer from low performances. Our objective is to introduce single-trial connectivity feature based BCI system for cognition imagery (CI) based tasks wherein the associated brain regions are located relatively far away as compared to those for MI. APPROACH: We implemented time-domain partial Granger causality (PGC) for the estimation of the connectivity features in a BCI setting. The proposed hypothesis has been verified with two publically available datasets involving MI and CI tasks. MAIN RESULTS: The results support the conclusion that connectivity based features can provide a better performance than a classical signal processing framework based on bandpass features coupled with spatial filtering for CI tasks, including word generation, subtraction, and spatial navigation. These results show for the first time that connectivity features can provide a reliable performance for imagery-based BCI system. SIGNIFICANCE: We show that single-trial connectivity features for mixed imagery tasks (i.e. combination of CI and MI) can outperform the features obtained by current state-of-the-art method and hence can be successfully applied for BCI applications.

Best practice for single-trial detection of event-related potentials: Application to brain-computer interfaces.

The detection of event-related potentials (ERPs) in the electroencephalogram (EEG) signal is a fundamental component in non-invasive brain-computer interface (BCI) research, and in modern cognitive neuroscience studies. Whereas the grand average response across trials provides an estimation of essential characteristics of a brain-evoked response, an estimation of the differences between trials for a particular type of stimulus can provide key insight about the brain dynamics and possible origins of the brain response. The research in ERP single-trial detection has been mainly driven by applications in biomedical engineering, with an interest from machine learning and signal processing groups that test novel methods on noisy signals. Efficient single-trial detection techniques require processing steps that include temporal filtering, spatial filtering, and classification. In this paper, we review the current state-of-the-art methods for single-trial detection of event-related potentials with applications in BCI. Efficient single-trial detection techniques should embed simple yet efficient functions requiring as few hyper-parameters as possible. The focus of this paper is on methods that do not include a large number of hyper-parameters and can be easily implemented with datasets containing a limited number of trials. A benchmark of different classification methods is proposed on a database recorded from sixteen healthy subjects during a rapid serial visual presentation task. The results support the conclusion that single-trial detection can be achieved with an area under the ROC curve superior to 0.9 with less than ten sensors and 20 trials corresponding to the presentation of a target. Whereas the number of sensors is not a key element for efficient single-trial detection, the number of trials must be carefully chosen for creating a robust classifier.

Single-Trial Detection With Magnetoencephalography During a Dual-Rapid Serial Visual Presentation Task.

GOAL: The detection of brain responses corresponding to the presentation of a particular class of images is a challenge in brain-machine interface. Current systems based on the detection of brain responses during rapid serial visual presentation (RSVP) tasks possess advantages for both healthy and disabled people, as they are gaze independent and can offer a high throughput. METHODS: We propose a novel paradigm based on a dual-RSVP task that assumes a low target probability. Two streams of images are presented simultaneously on the screen, the second stream is identical to the first one, but delayed in time. Participants were asked to detect images containing a person. They follow the first stream until they see a target image, then change their attention to the second stream until the target image reappears, finally they change their attention back to the first stream. RESULTS: The performance of single-trial detection was evaluated on both streams and their combination of the decisions with signal recorded with magnetoencephalography (MEG) during the dual-RSVP task. We compare classification performance across different sets of channels (magnetometers, gradiometers) with a BLDA classifier with inputs obtained after spatial filtering. CONCLUSION: The results suggest that single-trial detection can be obtained with an area under the ROC curve superior to 0.95, and that an almost perfect accuracy can be obtained with some subjects thanks to the combination of the decisions from two trials, without doubling the duration of the experiment. SIGNIFICANCE: The present results show that a reliable accuracy can be obtained with the MEG for target detection during a dual-RSVP task.

The effect of target and non-target similarity on neural classification performance: a boost from confidence.

Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

Semi-automatic ground truth generation using unsupervised clustering and limited manual labeling: Application to handwritten character recognition.

For training supervised classifiers to recognize different patterns, large data collections with accurate labels are necessary. In this paper, we propose a generic, semi-automatic labeling technique for large handwritten character collections. In order to speed up the creation of a large scale ground truth, the method combines unsupervised clustering and minimal expert knowledge. To exploit the potential discriminant complementarities across features, each character is projected into five different feature spaces. After clustering the images in each feature space, the human expert labels the cluster centers. Each data point inherits the label of its cluster's center. A majority (or unanimity) vote decides the label of each character image. The amount of human involvement (labeling) is strictly controlled by the number of clusters - produced by the chosen clustering approach. To test the efficiency of the proposed approach, we have compared, and evaluated three state-of-the art clustering methods (k-means, self-organizing maps, and growing neural gas) on the MNIST digit data set, and a Lampung Indonesian character data set, respectively. Considering a k-nn classifier, we show that labeling manually only 1.3% (MNIST), and 3.2% (Lampung) of the training data, provides the same range of performance than a completely labeled data set would.

Optimization of Single-Trial Detection of Event-Related Potentials Through Artificial Trials.

GOAL: Many brain-computer interface (BCI) classification techniques rely on a large number of labeled brain responses to create efficient classifiers. A large database representing all of the possible variability in the signal is impossible to obtain in a short period of time, and prolonged calibration times prevent efficient BCI use. We propose to improve BCIs based on the detection of event-related potentials (ERPs) in two ways. METHODS: First, we increase the size of the training database by considering additional deformed trials. The creation of the additional deformed trials is based on the addition of Gaussian noise, and on the variability of the ERP latencies. Second, we exploit the variability of the ERP latencies by combining decisions across multiple deformed trials. These new methods are evaluated on data from 16 healthy subjects participating in a rapid serial visual presentation task. RESULTS: The results show a significant increase in the performance of single-trial detection with the addition of artificial trials, and the combination of decisions obtained from altered trials. When the number of trials to train a classifier is low, the proposed approach allows us improve performance from an AUC of 0.533±0.080 to 0.905±0.053. This improvement represents approximately an 80% reduction in classification error. CONCLUSION: These results demonstrate that artificially increasing the training dataset leads to improved single-trial detection. SIGNIFICANCE: Calibration sessions can be shortened for BCIs based on ERP detection.

Towards increasing the number of commands in a hybrid brain-computer interface with combination of gaze and motor imagery.

Non-invasive brain-computer interface (BCI) provides a novel means of communication. This can be achieved by measuring electroencephalogram (EEG) signal over the sensory motor cortex of a person performing motor imagery (MI) tasks. However, the performance of BCI remains currently too low to be of wide practical use. A hybrid BCI system could improve the performance by combining two or more modalities such as eye tracking, and the detection of brain activity responses. In this paper, first, we propose a simultaneous hybrid BCI that combines an event-related de-synchronization (ERD) BCI and an eye tracker. Second, we aim to further improve performance by increasing the number of commands (i.e., the number of choices accessible to the user). In particular, we show a significant improvement in performance for a simultaneous gaze-MI system using a total of eight commands. The experimental task requires subjects to search for spatially located items using gaze, and select an item using MI signals. This experimental task studied visuomotor compatible and incompatible conditions. As incorporating incompatible conditions between gaze direction and MI can increase the number of choices in the hybrid BCI, our experimental task includes single-trial detection for average, compatible and incompatible conditions, using seven different classification methods. The mean accuracy for MI, and the information transfer rate (ITR) for the compatible condition is found to be higher than the average and the incompatible conditions. The results suggest that gaze-MI hybrid BCI systems can increase the number of commands, and the location of the items should be taken into account for designing the system.

Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering.

Accurate detection of single-trial event-related potentials (ERPs) in the electroencephalogram (EEG) is a difficult problem that requires efficient signal processing and machine learning techniques. Supervised spatial filtering methods that enhance the discriminative information in EEG data are commonly used to improve single-trial ERP detection. We propose a convolutional neural network (CNN) with a layer dedicated to spatial filtering for the detection of ERPs and with training based on the maximization of the area under the receiver operating characteristic curve (AUC). The CNN is compared with three common classifiers: 1) Bayesian linear discriminant analysis; 2) multilayer perceptron (MLP); and 3) support vector machines. Prior to classification, the data were spatially filtered with xDAWN (for the maximization of the signal-to-signal-plus-noise ratio), common spatial pattern, or not spatially filtered. The 12 analytical techniques were tested on EEG data recorded in three rapid serial visual presentation experiments that required the observer to discriminate rare target stimuli from frequent nontarget stimuli. Classification performance discriminating targets from nontargets depended on both the spatial filtering method and the classifier. In addition, the nonlinear classifier MLP outperformed the linear methods. Finally, training based AUC maximization provided better performance than training based on the minimization of the mean square error. The results support the conclusion that the choice of the systems architecture is critical and both spatial filtering and classification must be considered together.

Correction: cecotti, h. And rivet, B. Subject combination and electrode selection in cooperative brain-computer interface based on event related potentials. Brain sci. 2014, 4, 335-355.

The authors wish to make the following correction to this paper (Cecotti, H.; Rivet, B. Subject Combination and Electrode Selection in Cooperative Brain-Computer Interface Based on Event Related Potentials. Brain Sci. 2014, 4, 335-355). Dut to an error the reference number in the original published paper were not shown. The former main text should be replaced as below.

Subject combination and electrode selection in cooperative brain-computer interface based on event related potentials.

New paradigms are required in Brain-Computer Interface (BCI) systems for the needs and expectations of healthy people. To solve this issue, we explore the emerging field of cooperative BCIs, which involves several users in a single BCI system. Contrary to classical BCIs that are dependent on the unique subject's will, cooperative BCIs are used for problem solving tasks where several people shall be engaged by sharing a common goal. Similarly as combining trials over time improves performance, combining trials across subjects can significantly improve performance compared with when only a single user is involved. Yet, cooperative BCIs may only be used in particular settings, and new paradigms must be proposed to efficiently use this approach. The possible benefits of using several subjects are addressed, and compared with current single-subject BCI paradigms. To show the advantages of a cooperative BCI, we evaluate the performance of combining decisions across subjects with data from an event-related potentials (ERP) based experiment where each subject observed the same sequence of visual stimuli. Furthermore, we show that it is possible to achieve a mean AUC superior to 0.95 with 10 subjects and 3 electrodes on each subject, or with 4 subjects and 6 electrodes on each subject. Several emerging challenges and possible applications are proposed to highlight how cooperative BCIs could be efficiently used with current technologies and leverage BCI applications.